Current computer hardware allows for the display of volume data with different rendering techniques simultaneously and in real time. For a certain medical diagnostic task in the clinical routine, a hanging protocol defines how the data are reformatted and arranged on the screen. For some examinations, e.g., in mammography, the hanging protocol is highly standardized, whereas, e.g., in vascular examinations, more often customized hanging protocols are preferred. Frequently, Multi-Planar Reformatting (MPR) is the technique of choice to provide sectional renderings. With a Curved Planar Reformation (CPR) the whole extent of a tubular structure is displayed within a single image.
These two-dimensional (2D) renderings are often accompanied by a Direct Volume Rendering (DVR) technique like ray casting. The examination of a structure in its three-dimensional (3D) setting often provides better insights into contextual information. A typical hanging protocol arranges a side-by-side presentation of different views of the volumetric data. The physician performs different interactions during the examination of the data. Examples of frequently recurring interactions are scrolling through slices (2D images), zooming, panning, labeling, windowing (2D/3D images) or viewpoint selection and clipping (3D images).
The synchronization of the different views is quite challenging because it is not trivial to determine if an interaction in one view leads to changes within another view.
In European patent application No. 07118075.6 and scientific publications of Kohlmann et al., see P. Kohlmann, S. Bruckner, A. Kanitsar, and M. E. Gröller. LiveSync: Deformed viewing spheres for knowledge-based navigation. IEEE Transactions on Visualization and Computer Graphics, 13(6):1544-1551, 2007 and P. Kohlmann, S. Bruckner, A. Kanitsar, and M. E. Gröller. LiveSync++: Enhancements of an interaction metaphor. In Proceedings of Graphics Interface 2008, pages 81-88, 2008, solutions for the live synchronization of a 2D slice view and a 3D volumetric view were provided, wherein viewing parameters for the 3D view are derived automatically from a picking on the anatomical structure of interest on the 2D slice. The viewing parameters are viewpoint, zoom factor, clipping planes and transfer function setup.
In medical visualization some techniques have been developed to ease the interaction with multiple views of a certain data set. T. Götzelmann, P.-P. Vázquez, K. Hartmann, T. Germer, A. Nürnberger, and T. Strothotte. Mutual text-image queries. In Proceedings of Spring Conference on Computer Graphics 2007, pages 181-188, 2007 presented an approach where 3D visualizations are linked with textual descriptions, e.g., from medical textbooks. This approach focuses on an educational purpose and supports students to learn the terminology and to understand textual descriptions of complex objects. Related to this work, IBM, see The Official IBM Website, September 2008, is currently developing the Anatomic and Symbolic Mapper Engine (ASME), R. N. Charette. Visualizing electronic health records with “Google-Earth for the body”. IEEE Spectrum Online, January 2008. Available online at http://www.spectrum.ieee.org/jan08/5854/, September 2008. This technology uses a 3D model of the human body which is linked to medical records. Whenever the doctor clicks on a certain part of the body, a search of the medical records is triggered to extract the relevant information. The Medical Exploration Toolkit (METK), Medical Exploration Toolkit. Available online at http://www.metk.net, September 2008, presented by C. Tietjen, K. Mühler, F. Ritter, O. Konrad, M. Hindennach, and B. Preim. METK—The medical exploration toolkit. In Proceedings of Bildverarbeitung für die Medizin 2008, pages 407-411, 2008, bundles various concepts for loading, visualizing and exploring segmented medical data sets. Critical distances to pathological structures are computed and displayed in a synchronized manner in 2D and 3D. The integrated LIFTCHART, see C. Tietjen, B. Meyer, S. Schlechtweg, B. Preim, I. Hertel, and G. Strauβ. Enhancing slice-based visualizations of medical volume data, In Proceedings of IEEE/Eurographics Symposium on Visualization 2006, pages 123-130, 2006, displays the overall extents of structures within the volume in a narrow frame as color bars. A structure can be selected in the LIFTCHART and the corresponding slice is displayed in the slice viewer. Another clinical application they integrated is the NECKSURGERYPLANNER, C. Tietjen, B. Preim, I. Hertel, and G. Strauβ, A software-assistant for pre-operative planning and visualization of neck dissections. In CURAC 2006, pages 176-177, 2006. Segmented structures can be enabled and disabled by a textual representation. They are synchronously highlighted in the 3D and 2D views to support operation planning for neck dissections.
Some research has concentrated on the extraction of certain anatomical structures. Especially for the detection of curvilinear structures multi-scale filtering approaches are well-known. Vessel enhancement filters based on eigenvalue analysis of the Hessian matrix have been proposed, e.g., C. Lorenz, I.-C. Carlsen, T. M. Buzug, C. Fassnacht, and J. Weese, Multi-scale line segmentation with automatic estimation of width, contrast and tangential direction in 2D and 3D medical images. In Proceedings of the First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medical Robotics and Computer-Assisted Surgery, pages 233-242. Springer-Verlag, 1997; Y. Sato, C.-F. Westin, A. Bhalerao, S. Nakajima, N. Shiraga, S. Tamura, and R Kikinis, Tissue classification based on 3D local intensity structures for volume rendering, IEEE Transactions on Visualization and Computer Graphics, 6(2):160-180, 2000; and A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever, Multiscale vessel enhancement filtering, In Proceedings of MICCAI 1998, pages 130-137, 1998. H. Tek, D. Comaniciu, and J. P. Williams, Vessel detection by mean shift based ray propagation, In Proceedings of IEEE Workshop on Mathematical Methods in Biomedical Image Analysis 2001, pages 228-235, 2001, presented an approach which focuses on the segmentation of vessel cross sections. A single click inside the vessel on a slice initiates mean shift based ray propagation to detect the boundary of the vessel.
Other anatomical tubular structures are, e.g., the airway and the aorta. J. Tschirren, E. A. Hoffman, G. McLennan, and M. Sonka, Intrathoracic airway trees: Segmentation and airway morphology analysis from low-dose CT scans, IEEE Transactions on Medical Imaging, 24(12):1529-1539, 2005, presented an airway segmentation algorithm based on fuzzy connectivity. Their method uses small adaptive regions of interest which follow the airway branches during the segmentation process. T. Kovács, P. C. Cattin, H. Alkadhi, S. Wildermuth, and G. Székely, Automatic segmentation of the vessel lumen from 3D CTA images of aortic dissection, In Proceedings of Bildverarbeitung für die Medizin 2006, pages 161-165, 2006, developed a system for automatic segmentation of the entire aorta without any user interaction for treatment planning of aortic dissections. The segmentation is based on a Hough transformation to detect the approximate circular shape of the aorta. To fit this shape more closely to the actual contour of the aortic lumen an elastic mass-spring deformable model is utilized. An interesting concept for the detection of tubular objects in medical imaging is the Gradient Vector Flow (GVF) snake introduced by C. Xu and J. L. Prince. Gradient vector flow: A new external force for snakes. In Proceedings of the Conference on Computer Vision and Pattern Recognition 1997, pages 66-71, 1997. This method first calculates a field of forces (GVF forces) over the image domain. These forces drive the snake to fit to the boundaries of an object. C. Bauer and H. Bischof in A novel approach for detection of tubular objects and its application to medical image analysis, In Proceedings of the 30th DAGM Symposium on Pattern Recognition, pages 163-172, 2008 and C. Bauer and H. Bischof in Extracting curve skeletons from gray value images for virtual endoscopy, In Proceedings of the 4th International Workshop on Medical Imaging and Augmented Reality, pages 393-402, 2008 utilize the properties of the GVF for the detection of tubular objects and the extraction of curve skeletons, e.g., for virtual endoscopy. They argue that conventional tube detection or line filters, which use local derivatives at multiple scales have problems with undesired diffusion of nearby objects. Their GVF-based method allows an edge-preserving diffusion of gradient information. M. M. Malik, T. Möller, and M. E. Grolier, Feature peeling. In Proceedings of Graphics Interface 2007, pages 273-280, 2007, presented a rendering algorithm called feature peeling. They analyze peaks and valleys of intensity ray profiles for a given viewpoint to detect features inside the volume data. By classifying a number of feature layers it is possible to scroll through the layers to inspect various structures.